Electronic devices with gaze detection capabilities

ABSTRACT

An electronic device may have gaze detection capabilities that allow the device to detect when a user is looking at the device. The electronic device may implement a power management scheme using the results of gaze detection operations. When the device detects that the user has looked away from the device, the device may dim a display screen and may perform other suitable actions. The device may pause a video playback operation when the device detects that the user has looked away from the device. The device may resume the video playback operation when the device detects that the user is looking towards the device. Gaze detector circuitry may be powered down when sensor data indicates that gazed detection readings will not be reliable or are not needed.

BACKGROUND

This invention relates generally to electronic devices, and moreparticularly, to electronic devices such as portable electronic devicesthat have gaze detection capabilities.

Electronic devices such as portable electronic devices are becomingincreasingly popular. Examples of portable devices include handheldcomputers, cellular telephones, media players, and hybrid devices thatinclude the functionality of multiple devices of this type. Popularportable electronic devices that are somewhat larger than traditionalhandheld electronic devices include laptop computers and tabletcomputers.

To satisfy consumer demand for small form factor portable electronicdevices, manufacturers are continually striving to reduce the size ofcomponents that are used in these devices. For example, manufacturershave made attempts to miniaturize the batteries used in portableelectronic devices.

An electronic device with a small battery has limited battery capacity.Unless care is taken to consume power wisely, an electronic device witha small battery may exhibit unacceptably short battery life. Techniquesfor reducing power consumption may be particularly important in wirelessdevices that support cellular telephone communications, because users ofcellular telephone devices often demand long “talk” times.

Conventional portable electronic devices use various techniques forreducing their power consumption. Because display screens in electronicdevices can consume relatively large amounts of power, powerconservation techniques in portable electronic devices with displayscreens typically involve turning off the display screens at particulartimes. Unfortunately, conventional power conservation techniques mayturn off display screens at inappropriate times, thereby interferingwith a user's ability to interact with a device. Conventional techniquesmay also leave display screens on at inappropriate times, wastingvaluable battery power.

It would therefore be desirable to be able to provide improved ways inwhich to conserve power in electronic devices.

SUMMARY

An electronic device is provided that may have gaze detectioncapabilities. One or more gaze detection sensors such as a camera may beused by the electronic device to determine whether a user's gaze isdirected towards the electronic device (e.g., whether the user of theelectronic device is looking at the electronic device). In particular,the electronic device may use gaze detection sensors to determinewhether or not the user is looking at a display portion of theelectronic device.

In an illustrative embodiment, the electronic device may have powermanagement capabilities that are used to help conserve power. Theelectronic device may operate in two or more operating modes. Oneoperation mode may be used to optimize performance. Another operatingmode may help to extend battery life. The electronic device may useresults from gaze detection operations to determine an appropriate modein which to operate the electronic device.

For example, the electronic device may operate in an active mode whenthe electronic device determines, using gaze detection sensors, that theuser's gaze is directed towards the electronic device and may operate inone or more standby modes when the device determines that the user'sgaze is not directed towards the electronic device. When the electronicdevice is operating in one of the standby modes, circuitry andcomponents such as a display screen, touch screen components, gazedetection components, and a central processing unit or CPU in theelectronic device may be powered down or operated in a low-power mode tominimize power consumption in the electronic device.

With one suitable arrangement, when the electronic device is in theactive mode and detects that the user has looked away from the device,the electronic device may dim or turn off a display screen. If desired,the electronic device can dim the display screen to a standby brightnesslevel after the device has determined that the user has looked away fromthe device. After a given period of time has elapsed in which no userinput has been received by the electronic device, the electronic devicecan turn off the display screen to conserve power. When the electronicdevice detects that the user's gaze is directed towards the electronicdevice, the electronic device may enter the active mode and return thedisplay screen to an active brightness level (e.g., turn on the displayscreen or brighten the display screen to the active brightness level).

If desired, the electronic device may be performing an operation, whilein the active mode, that is uninterrupted when the electronic deviceswitches to operating in one of the standby modes. For example, theelectronic device may be performing a music playback operation while inthe active mode and, when the electronic device detects the user's gazeis not directed towards the electronic device, the electronic device mayenter one of the standby modes without interrupting the music playbackoperation.

With one suitable arrangement, the electronic device may interrupt anoperation when the electronic device begins operating in one of thestandby mode. For example, the electronic device may be performing avideo playback operation while in the active mode. In this example, whenthe electronic device detects that the user's gaze is no longer directedtowards the electronic device, the electronic device may enter one ofthe standby modes, dim the display screen that was being used for thevideo playback operation, and pause the video playback operation. Ifdesired, the electronic device may resume the video playback operationwhen it detects that the user has redirected their gaze towards theelectronic device (e.g., towards the video screen).

In an illustrative embodiment, the electronic device may use readingsfrom sensors such as proximity sensors, ambient light sensors, andmotion sensors such as accelerometers to determine whether or not toperform gaze detection operations. For example, the electronic devicemay suspend gaze detection operations whenever a proximity sensor,ambient light sensor, or accelerometer indicates that gaze detectionoperations are inappropriate (e.g., because of an object in closeproximity with the electronic device, insufficient ambient light forgaze detection sensors to detect the user's gaze, excessive vibrationwhich may degrade the performance of gaze detection sensors, etc.).

An advantage of powering down the display is that a powered down displaycan help to prevent information on the display from being viewed by anunauthorized viewer. It may therefore be helpful to turn off a displaywhen the lack of a user's gaze indicates that the user is not present toguard the device.

Further features of the invention, its nature and various advantageswill be more apparent from the accompanying drawings and the followingdetailed description of the preferred embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view of an illustrative portable electronicdevice that may have gaze detection capabilities in accordance with anembodiment of the present invention.

FIG. 2 is a schematic diagram of an illustrative portable electronicdevice that may have gaze detection capabilities in accordance with anembodiment of the present invention.

FIG. 3 is a state diagram of illustrative operating modes of anillustrative electronic device with gaze detection capabilities inaccordance with an embodiment of the present invention.

FIG. 4 is a state diagram of illustrative operating modes of anillustrative electronic device with gaze detection capabilities during amusic playback operation in accordance with an embodiment of the presentinvention.

FIG. 5 is a state diagram of illustrative operating modes of anillustrative electronic device with gaze detection capabilities andactivity detection capabilities in accordance with an embodiment of thepresent invention.

FIG. 6 is a state diagram of illustrative operating modes of anillustrative electronic device with gaze detection capabilities during avideo playback operation in accordance with an embodiment of the presentinvention.

FIG. 7 is a state diagram of illustrative operating modes of anillustrative electronic device with gaze detection and touch screeninput capabilities in accordance with an embodiment of the presentinvention.

FIG. 8 is a state diagram of illustrative operating modes of anillustrative electronic device with gaze detection capabilities inaccordance with an embodiment of the present invention.

FIG. 9 is a state diagram of illustrative operating modes of anillustrative electronic device with gaze detection capabilities andsensors such as environment sensors in accordance with an embodiment ofthe present invention.

FIG. 10 is a flow chart of illustrative steps involved in reducing powerto displays in an electronic device in accordance with an embodiment ofthe present invention.

DETAILED DESCRIPTION

The present invention relates generally to electronic devices, and moreparticularly, to electronic devices such as portable electronic devicesthat have gaze detection capabilities.

With one suitable arrangement, an electronic device with gaze detectioncapabilities may have the ability to determine whether a user's gaze iswithin a given boundary without resolving the specific location of theuser's gaze within that boundary. The electronic device, as an example,may be able to detect whether a user's gaze is directed towards adisplay associated with the device. With another suitable arrangement,an electronic device may have gaze tracking capabilities. Gaze trackingcapabilities allow the electronic device to determine not only whetheror not a user's gaze is directed towards a display associated with thedevice but also which portion of the display the user's gaze is directedtowards.

An electronic device may be used to detect a user's gaze and adjust itsbehavior according to whether or not the user's gaze is detected. Forexample, the electronic device may be able to detect whether or not theuser is looking at the device and adjust power management settingsaccordingly. With one suitable arrangement, the electronic device maydelay turning device components off (e.g., components which wouldotherwise be turned off as part of a power management scheme) while theuser's gaze is directed towards the device and the electronic device mayaccelerate the shutdown of device components when the user's gaze is notdetected. For example, when the user's gaze is detected, a device with adisplay may keep the display at normal brightness rather than dimmingthe display and, when the device detects the user is no longing lookingat the device, the device may dim or turn off the display. This type ofarrangement may be especially beneficial in situations in which the useris not actively controlling the electronic device (e.g., the user is notpressing buttons or supplying touch screen inputs) but is stillinteracting with the electronic device (e.g., the user is reading texton the display, watching video on the display, etc.). An advantage ofturning off the display when the user is not looking at the display isthis may help prevent unauthorized users from viewing information on thedisplay, thereby enhancing device security.

Electronic devices that have gaze detection capabilities may be portableelectronic devices such as laptop computers or small portable computersof the type that are sometimes referred to as ultraportables. Portableelectronic devices may also be somewhat smaller devices. Examples ofsmaller portable electronic devices include wrist-watch devices, pendantdevices, headphone and earpiece devices, and other wearable andminiature devices. With one suitable arrangement, the portableelectronic devices may be wireless electronic devices.

The wireless electronic devices may be, for example, handheld wirelessdevices such as cellular telephones, media players with wirelesscommunications capabilities, handheld computers (also sometimes calledpersonal digital assistants), global positioning system (GPS) devices,and handheld gaming devices. The wireless electronic devices may also behybrid devices that combine the functionality of multiple conventionaldevices. Examples of hybrid portable electronic devices include acellular telephone that includes media player functionality, a gamingdevice that includes a wireless communications capability, a cellulartelephone that includes game and email functions, and a portable devicethat receives email, supports mobile telephone calls, has music playerfunctionality, and supports web browsing. These are merely illustrativeexamples.

An illustrative portable electronic device in accordance with anembodiment of the present invention is shown in FIG. 1. User device 10may be any suitable electronic device such as a portable or handheldelectronic device. Device 10 of FIG. 1 may be, for example, a handheldelectronic device that supports 2G and/or 3G cellular telephone and datafunctions, global positioning system capabilities or other satellitenavigation capabilities, and local wireless communications capabilities(e.g., IEEE 802.11 and Bluetooth®) and that supports handheld computingdevice functions such as internet browsing, email and calendarfunctions, games, music player functionality, etc.

Device 10 may have a housing 12. Display 16 may be attached to housing12 using bezel 14. Display 16 may be a touch screen liquid crystaldisplay (as an example). Display 16 may have pixels that can becontrolled individually in connection with power consumptionadjustments. For example, in an organic light emitting diode (OLED)display, power can be reduced by making full and/or partial brightnessreductions to some or all of the pixels. Display 16 may be formed from apanel subsystem and a backlight subsystem. For example, display 16 mayhave a liquid crystal display (LCD) panel subsystem and a light emittingdiode or fluorescent tube backlight subsystem. In backlight subsystemsthat contain individually controllable elements such as light emittingdiodes, the brightness of the backlight elements may be selectivelycontrolled. For example, the brightness of some of the backlightelements may be reduced while the other backlight elements remain fullypowered. In backlight subsystems that contain a single backlightelement, the power of the single element may be partially or fullyreduced to reduce power consumption. It may also be advantageous to makepower adjustments to the circuitry that drives the LCD panel subsystem.

Display screen 16 (e.g., a touch screen) is merely one example of aninput-output device that may be used with electronic device 10. Ifdesired, electronic device 10 may have other input-output devices. Forexample, electronic device 10 may have user input control devices suchas button 19, and input-output components such as port 20 and one ormore input-output jacks (e.g., for audio and/or video). Button 19 maybe, for example, a menu button. Port 20 may contain a 30-pin dataconnector (as an example). Openings 22 and 24 may, if desired, formspeaker and microphone ports. Speaker port 22 may be used when operatingdevice 10 in speakerphone mode. Opening 23 may also form a speaker port.For example, speaker port 23 may serve as a telephone receiver that isplaced adjacent to a user's ear during operation. In the example of FIG.1, display screen 16 is shown as being mounted on the front face ofhandheld electronic device 10, but display screen 16 may, if desired, bemounted on the rear face of handheld electronic device 10, on a side ofdevice 10, on a flip-up portion of device 10 that is attached to a mainbody portion of device 10 by a hinge (for example), or using any othersuitable mounting arrangement.

A user of electronic device 10 may supply input commands using userinput interface devices such as button 19 and touch screen 16. Suitableuser input interface devices for electronic device 10 include buttons(e.g., alphanumeric keys, power on-off, power-on, power-off, and otherspecialized buttons, etc.), a touch pad, pointing stick, or other cursorcontrol device, a microphone for supplying voice commands, or any othersuitable interface for controlling device 10. Buttons such as button 19and other user input interface devices may generally be formed on anysuitable portion of electronic device 10. For example, a button such asbutton 19 or other user interface control may be formed on the side ofelectronic device 10. Buttons and other user interface controls can alsobe located on the top face, rear face, or other portion of device 10. Ifdesired, device 10 can be controlled remotely (e.g., using an infraredremote control, a radio-frequency remote control such as a Bluetooth®remote control, etc.).

If desired, device 10 may contain sensors such as a proximity sensor andan ambient light sensor. A proximity sensor may be used to detect whendevice 10 is close to a user's head or other object. An ambient lightsensor may be used to make measurements of current light levels.

Device 10 may have a camera or other optical sensor such as camera 30that can be used for gaze detection operations. Cameras used for gazedetection may, for example, be used by device 10 to capture images of auser's face that are processed by device 10 to detect where the user'sgaze is directed. Camera 30 may be integrated into housing 12. Whileshown as being formed on the top face of electronic device 10 in theexample of FIG. 1, cameras such as camera 30 may generally be formed onany suitable portion of electronic device 10. For example, camera 30 maybe mounted on a flip-up portion of device 10 that is attached to a mainbody portion of device 10 by a hinge or may be mounted between theflip-up portion of device 10 and the main body portion of device 10(e.g., in the hinge region between the flip-up portion and the main bodyportion such that the camera can be used regardless of whether thedevice is flipped open or is closed). Device 10 may also have additionalcameras (e.g., device 10 may have camera 30 on the top face of device 10for gaze detection operations and another camera on the bottom face ofdevice 10 for capturing images and video).

If desired, the gaze detection functions of camera 30 may be implementedusing an optical sensor that has been optimized for gaze detectionoperations. For example, camera 30 may include one or more lightemitting diodes (LED's) and an optical sensor capable of detectingreflections of light emitted from the LEDs off of the users' eyes whenthe users are gazing at device 10. The light emitting diodes may emit amodulated infrared light and the optical sensor may be synchronized todetect reflections of the modulated infrared light, as an example. Ingeneral, any suitable gaze detection image sensor and circuitry may beused for supporting gaze detection operations in device 10. The use ofcamera 30 is sometimes described herein as an example.

A schematic diagram of an embodiment of an illustrative portableelectronic device such as a handheld electronic device is shown in FIG.2. Portable device 10 may be a mobile telephone, a mobile telephone withmedia player capabilities, a handheld computer, a remote control, a gameplayer, a global positioning system (GPS) device, a laptop computer, atablet computer, an ultraportable computer, a hybrid device thatincludes the functionality of some or all of these devices, or any othersuitable portable electronic device.

As shown in FIG. 2, device 10 may include storage 34. Storage 34 mayinclude one or more different types of storage such as hard disk drivestorage, nonvolatile memory (e.g., flash memory or otherelectrically-programmable-read-only memory), volatile memory (e.g.,battery-based static or dynamic random-access-memory), etc.

Processing circuitry 36 may be used to control the operation of device10. Processing circuitry 36 may be based on a processor such as amicroprocessor and other suitable integrated circuits. With one suitablearrangement, processing circuitry 36 and storage 34 are used to runsoftware on device 10, such as gaze detection applications, internetbrowsing applications, voice-over-internet-protocol (VOIP) telephonecall applications, email applications, media playback applications,navigation functions, map functions, operating system functions, powermanagement functions, etc. Processing circuitry 36 and storage 34 may beused in implementing suitable communications protocols. Communicationsprotocols that may be implemented using processing circuitry 36 andstorage 34 include internet protocols, wireless local area networkprotocols (e.g., IEEE 802.11 protocols—sometimes referred to as Wi-Fi®),protocols for other short-range wireless communications links such asthe Bluetooth® protocol, protocols for handling 3G communicationsservices (e.g., using wide band code division multiple accesstechniques), 2G cellular telephone communications protocols, etc. Ifdesired, processing circuitry 36 may operate in a reduced power mode(e.g., circuitry 36 may be suspended or operated at a lower frequency)when device 10 enters a suitable standby mode.

Input-output devices 38 may be used to allow data to be supplied todevice 10 and to allow data to be provided from device 10 to externaldevices. Display screen 16, camera 30, button 19, microphone port 24,speaker port 22, and dock connector port 20 are examples of input-outputdevices 38. In general, input-output devices 38 may include any suitablecomponents for receiving input and/or providing output from device 10.For example, input-output devices 38 can include user input-outputdevices 40 such as buttons, touch screens, joysticks, click wheels,scrolling wheels, touch pads, key pads, keyboards, microphones, etc. Auser can control the operation of device 10 by supplying commandsthrough user input devices 40. Input-output device 38 may includesensors such as proximity sensors, ambient light sensors, orientationsensors, proximity sensors, and any other suitable sensors.

Input-output devices 38 may include a camera such as integrated camera41 (e.g., a camera that is integrated into the housing of device 10) andcamera 30 of FIG. 1. Cameras such as camera 41 and camera 30 may be usedas part of a gaze detection system. For example, camera 41 may be usedby device 10 to capture images that are processed by a gaze detectionapplication running on processing circuitry 36 to determine whether ornot a user's gaze is directed towards the device. Cameras such as camera41 and camera 30 may, if desired, be provided with image stabilizationcapabilities (e.g., using feedback derived from an accelerometer,orientation sensor, or other sensor).

Display and audio devices 42 may include liquid-crystal display (LCD)screens or other screens, light-emitting diodes (LEDs), and othercomponents that present visual information and status data. Display andaudio devices 42 may also include audio equipment such as speakers andother devices for creating sound. Display and audio devices 42 maycontain audio-video interface equipment such as jacks and otherconnectors for external headphones and monitors.

Wireless communications devices 44 may include communications circuitrysuch as radio-frequency (RF) transceiver circuitry formed from one ormore integrated circuits, power amplifier circuitry, passive RFcomponents, antennas, and other circuitry for handling RF wirelesssignals. Wireless signals can also be sent using light (e.g., usinginfrared communications).

Device 10 can communicate with external devices such as accessories 46,computing equipment 48, and wireless network 49, as shown by paths 50and 51. Paths 50 may include wired and wireless paths. Path 51 may be awireless path. Accessories 46 may include headphones (e.g., a wirelesscellular headset or audio headphones) and audio-video equipment (e.g.,wireless speakers, a game controller, or other equipment that receivesand plays audio and video content), a peripheral such as a wirelessprinter or camera, etc.

Computing equipment 48 may be any suitable computer. With one suitablearrangement, computing equipment 48 is a computer that has an associatedwireless access point (router) or an internal or external wireless cardthat establishes a wireless connection with device 10. The computer maybe a server (e.g., an internet server), a local area network computerwith or without internet access, a user's own personal computer, a peerdevice (e.g., another portable electronic device 10), or any othersuitable computing equipment.

Wireless network 49 may include any suitable network equipment, such ascellular telephone base stations, cellular towers, wireless datanetworks, computers associated with wireless networks, etc.

A device such as device 10 that has gaze detection capabilities may usegaze detector data in implementing a power management scheme. As anexample, device 10 may operate in multiple modes to conserve power andmay utilize gaze detection operations to assist in determining anappropriate mode in which to operate.

With one suitable arrangement, the operational modes of device 10 mayinclude modes such as an active mode, a partial standby mode, and a fullstandby mode. In these and other operational modes, device 10 may adjustthe brightness of display 16 and may turn display 16 on or off wheneverappropriate in order to conserve power. For example, display 16 may beat an active brightness when device 10 is in the active mode, a standbybrightness when device 10 is in the partial standby mode, and may beturned off when device 10 is in the full standby mode. The standbybrightness may be somewhat dimmer than the active brightness. Generally,the power consumption of display 16 and therefore device 10 will bereduced when the brightness of display 16 is reduced and when display 16is turned off.

Consider, as an example, the scenario of FIG. 3. In mode 52 of FIG. 3,device 10 is in an active mode. In general, it is generally desirablefor device 10 to be in the active mode whenever a user is activelyinteracting with device 10. In particular, it is desirable for display16 to be at the active brightness level whenever the user's gaze isdirected towards display 16.

When device 10 is in the active mode, a display such as display 16 maybe turned on and may display an appropriate screen such as anapplication display screen at the active brightness level. The activebrightness level may be a configurable brightness level. For example,device 10 may receive input from a user to adjust the active brightnesslevel. In general, the active brightness level may be adjusted anywherebetween the maximum brightness and minimum brightness level display 16is capable of.

If desired, device 10 may be performing a music playback operation whendevice 10 is in the active mode. In the example of FIG. 3, the musicplayback operation may be occurring in the background of the operationof device 10 (e.g., device 10 may be performing the music playbackoperation while display 16 and user input device 40 are used by the userto perform additional tasks such as writing an e-mail, browsing the web,etc.).

While device 10 is in the active mode, device 10 may be performing gazedetection operations. For example, when device 10 is in the active mode,device 10 may be capturing images using camera 30 or other image sensingcomponents at regular intervals and maybe analyzing the images usinggaze detection software. Based on this analysis, the device candetermine whether the user's gaze is directed towards device 10 anddisplay 16. When device 10 is performing gaze detection operations,device 10 may be capturing images used for the gaze detection operationsat any suitable interval such as thirty times per second, ten times persecond, twice per second, once per second, every two seconds, every fiveseconds, upon occurrence of non-time-based criteria, combinations ofthese intervals, or at any other suitable time.

As illustrated by line 54, when device 10 detects that the user haslooked away, device 10 may dim display screen 16 and may enter partialstandby mode 56. Device 10 may detect that the user has diverted theirgaze away from device 10 and display 16 using a gaze detection sensorsuch as camera 30 and gaze detection software running on the hardware ofdevice 10. If desired, gaze detection processing may be offloaded tospecialized gaze detection circuitry (e.g., circuitry in a gazedetection chip or a camera controller).

In mode 56, device 10 is in a partial standby mode. In the partialstandby mode, the brightness level of display 16 may be reduced from anactive brightness level to a standby brightness level to reduce thepower consumption of device 10. When device 10 enters a standby modesuch as the partial standby mode, some operations running on device 10may be suspended or stopped and some operations may continue running.For example, a music playback operation may continue when device 10enters one of its standby modes while a web browsing application may besuspended. With this type of arrangement, when a user of device 10 islistening to music through the device while browsing the web on display16, device 10 can dim display 16 to conserve power whenever the userlooks away from display 16 while continuing to play back the music thatthe user is listening to without interruption.

As illustrated by line 58, when device 10 detects activity, device 10may brighten display screen 16 and may enter active mode 52. Device 10may enter active mode 52 in response to user activity such as buttonpress activity received through a button such as button 19 and inresponse to other activity such as network activity (e.g., activityreceived through a wired or wireless communications link). In this typeof arrangement, device 10 will enter the active mode whenever a userresumes interacting with device 10.

As illustrated by FIG. 4, device 10 may implement a power managementscheme that turns off display 16 based on gaze detection data. Inparticular, device 10 may turn off display 16 when the device detectsthat the user is not looking at display 16 (e.g., rather than merelydimming display 16 as in the example of FIG. 3).

In mode 60, device 10 is in an active mode. While device 10 is in theactive mode, device 10 may perform gaze detection operations. Becausedevice 10 is in the active mode, display 16 may be at an activebrightness level. With one suitable arrangement, when device 10 is inactive mode 60, device 10 may be displaying a screen with display 16that is of interest to the user but which does not demand the user'sconstant attention. For example, when device 10 is in mode 60, device 10may be displaying a screen such as a now playing screen associated witha music playback operation or a telephone information screen associatedwith a telephone operation (e.g., a new incoming call, a new outgoingcall, an active call, etc.). The now playing screen may, for example,include information about the music playback operation such as a trackname, album name, artist name, elapsed playback time, remaining playbacktime, album art, etc. and may include on-screen selectable options(e.g., when display 16 is a touch-screen display) such as play, pause,fast forward, rewind, skip ahead (e.g., to another audio track), skipback, stop, etc. A telephone information screen might includeinformation about a telephone operation such as a current call time, thetelephone number associated with a telephone call, a contact nameassociated with the telephone call, and an image associated with thetelephone call and may include on-screen selectable options such as akeypad to enter a telephone number, a call button, an end call button, ahold button, a speakerphone button, a mute button, an add call button, acontacts button, etc.

As illustrated by line 62, when device 10 detects that the user haslooked away from display 16, device 10 may turn off display 16 and mayenter standby mode 64. When device 10 is in standby mode 64, device 10may continue to perform background operations such as a music playbackoperation that was occurring before device 10 entered standby mode 64(e.g., before device 10 detected that the user's gaze was diverted awayfrom display 16). Because the application screen displayed in mode 60 isof secondary importance to the user, device 10 may turn off display 16completely when the user looks away without disrupting the user. Forexample, when a user is listening to an audio track and is also viewinginformation associated with the audio track on a now playing screen,device 10 can turn off display 16 when the user looks away, whilecontinuing an audio playback operation. The user's primary use of device10 (listening to music) is not interrupted, even though the secondaryuse of device 10 (viewing the now playing screen) has been halted.

In mode 64, device 10 is in a standby mode. In standby mode 64, display16 may be turned off by device 10 to conserve power. When device 10enters standby mode 64, suitable components of device 10 may be powereddown (if desired). For example, in mode 64, the power consumption ofprocessing circuitry 36 may be reduced (e.g., by operating fewerprocessor cores, by reducing the computing frequency of circuitry 36,etc.). With one suitable arrangement, an operation such as a musicplayback operation or a telephone call may continue when device 10 is inmode 64.

As illustrated by line 66, when device 10 detects activity such as useractivity, device 10 may enter active mode 60 and turn on display 16.Device 10 may enter active mode 60 in response to any suitable activitysuch as button press activity, network activity, and gaze detectionactivity (e.g., when device 10 detects that the user has directed theirgaze towards device 10).

As shown in the example of FIG. 5, device 10 may implement a powermanagement scheme that is responsive to gaze detection data and otherinput data (e.g., user input, network input, etc.). In the powermanagement scheme illustrated in FIG. 5, device 10 can switch between anactive mode, a partial standby mode, and a standby mode. Device 10 maypower down hardware components and suspend or slow down softwareoperations depending on the mode in which device 10 is operating. Forexample, when device 10 is in either of the standby modes, device 10 mayreduce the number of processing cores utilized by circuitry 36 and/ormay reduce the processing frequency (clock rate) of circuitry such ascircuitry 36. With one suitable arrangement, device 10 may turn display16 on at an active brightness level in the active mode, dim display 16to a standby brightness level in the partial standby mode, and turndisplay 16 off in the standby mode.

In mode 68, device 10 is in an active mode. While device 10 is in theactive mode, device 10 may perform gaze detection operations. Display 16may be at the active brightness level while device 10 is in active mode68. In mode 68, device 10 may be displaying an application displayscreen such as a home page, a music playback application screen, a webbrowsing application screen, an email application screen, etc. Ifdesired, device 10 may also be performing a music playback operationwhile in mode 68 (e.g., device 10 may be performing the music playbackoperation as a background process as device 10 displays the applicationdisplay screen).

When device 10 detects that the user has looked away from display 16(e.g., using a gaze detection sensor such as camera 30), device 10 maydim display 16 and enter partial standby mode 72, as illustrated by line70.

In mode 72, device 10 is in a partial standby mode. In partial standbymode 72, device 10 may dim display 16 to a partial standby brightnesslevel to conserve power and, if desired, may place other components suchas processing circuitry, wireless transceiver circuitry, etc. in astandby mode to conserve power. Certain operations may continue whendevice 10 enters mode 72. For example, a music playback operation or atelephone call may continue uninterrupted when device 10 enters mode 72.

Device 10 may perform gaze detection operations while in mode 72. Forexample, device 10 may continually capture images using camera 30 atregular intervals and may analyze the captured images using gazedetection software to determine whether the user's gaze has returned todevice 10 and display 16. If desired, the rate at which device 10captures and processes images for gaze detection operations while inmode 72 may be reduced relative to the rate at which gaze detectionimages are captured and processed while device is in an active mode suchas mode 68 (e.g., device 10 may capture images at a rate of once every100 milliseconds, 250 milliseconds, 500 milliseconds, 1 second, etc. inmode 72 and once every 50 milliseconds, 125 milliseconds, 250milliseconds, 500 milliseconds, etc. in mode 68).

Device 10 may switch from partial standby mode 72 to active mode 68whenever appropriate. For example, when device 10 detects that a user'sgaze is directed towards display 16, device 10 may enter an active modesuch as mode 68 (e.g., as illustrated by line 75) and may brightendisplay 16 to the active brightness level. Device 10 may also enteractive mode 68 when device 10 detects activity such as user activityreceived through a button such as button 19 and network activityreceived through a wired or wireless communications link (e.g., asillustrated by line 74). In general, device 10 will enter active mode 68whenever a user resumes interacting with device 10 or device 10 needs torespond to network activity. Because device 10 enters active mode 68when device 10 detects that the user's gaze is directed towards display16 (e.g., as illustrated by line 75), the user of device 10 need notpress a button or provide other input to awaken device 10 from thepartial standby state. Instead, device 10 can automatically awaken(e.g., switch to active mode 68) when device 10 detects that the userhas directed their gaze towards display 16.

If desired, device 10 may operate in a standby mode such as standby mode76 in which display 16 is turned off. For example, when device 10 isoperating in partial standby mode 72 and no user activity is detectedfor a given period of time (e.g., within a period of time such as onesecond, two seconds, . . . , ten seconds, twenty seconds, thirtyseconds, etc.), device 10 may enter standby mode 76 and turn off display16. Device 10 may enter standby mode 76, as illustrated by line 79,after device 10 detects that the user has looked away (e.g., asillustrated by line 70) and after a given period of user inactivity haselapsed following the device's detection that the user looked away.

In standby mode 76, device 10 may operate with display 16 turned off.Device 10 may place suitable components into standby. For example,device 10 may turn wireless transceiver circuitry off, reduce the powerconsumption of processing circuitry such as circuitry 36 (e.g., byturning off selected processing cores or lowering clock rates), turn offsensors such as proximity sensors, ambient light sensors, andaccelerometers, and may suspend or power down any other suitablecomponents. If desired, certain operations may continue when device 10enters and operates in standby mode 76. For example, a music playbackoperation or a telephone call may continue uninterrupted when device 10enters mode 76.

With the arrangement of FIG. 5, as long as device 10 detects that theuser's gaze is directed at the device (e.g., the user is looking atdisplay 16), device 10 may remain in active mode 68. Device 10 mayremain in active mode 68 even when no other user activity is received(e.g., when the user is not pressing a button such as button 19 orproviding user input through a touch screen such as touch screen display16). This type of arrangement may be beneficial when a user is utilizingdevice 10 without providing user input and would be inconvenienced bydevice 10 implementing power management techniques. Device 10 canoverride power management schemes such as dimming a display screen basedon results of gaze detection operations. For example, when device 10detects a user's gaze and is presenting the user with text or videothrough display 16, device 10 may override power management instructionsthat could otherwise reduce the power of display 16 to ensure thatdisplay 16 is not dimmed or turned off even though the user has notprovided direct user input.

If desired, device 10 may continue to perform gaze detection operationswhen operating in standby mode 76. As illustrated by dashed line 77,device 10 may switch from standby mode 76 to active mode 68 wheneverdevice 10 detects that a user's gaze is once again directed towardsdisplay 16.

As illustrated by line 78, when device 10 detects activity, device 10may switch from mode 76 to active mode 68 (e.g., device 10 may turn ondisplay 16 to the active brightness level). As an example, device 10 mayenter active mode 68 in response to user activity such as button pressactivity received through a button such as button 19 and in response toother activity such as network activity (e.g., activity received througha wired or wireless communications link).

As illustrated in FIG. 6, device 10 may implement a power managementscheme that utilizes gaze detection capabilities while executing a videoplayback operation (e.g., while playing video for a user). In the schemeillustrated by FIG. 6, device 10 can operate in an active mode (mode80), a pause standby mode (e.g., a partial standby mode such as mode84), and a standby mode (mode 90). With one suitable arrangement, device10 may be performing a video playback operation for a user when thedevice is in the active mode, device 10 may pause the video playbackoperation and dim an associated display screen when the user looks awayfrom the device and the device enters the pause standby mode, and device10 may turn off the display screen (e.g., the screen used for the videoplayback operation) if the user does not look back towards the devicewithin a given period of time and no other user activity is detected.

In active mode 80, device 10 is active. While device 10 is in the activemode, device 10 may perform gaze detection operations (e.g., usingcamera 30 and processing circuitry 36 to detect whether or not a user isgazing at display 16). While in mode 80, device 10 may perform a videoplayback operation. For example, device 10 may display video on display16 and may play audio associated with the video through a speaker suchas speaker 22 or through a headphone accessory such as accessory 46.Display 16 may display the video at an active brightness level (e.g.,display 16 may be at a relatively bright display level).

When device 10 detects that the user has looked away from display 16(e.g., using a gaze detection sensor such as camera 30), device 10 maydim display 16 and enter pause standby mode 84 as illustrated by line82. As part of entering pause standby mode 84, device 10 may pause thevideo playback operation of mode 80. Generally, when device 10 pausesthe video playback operation, device 10 will also pause an accompanyingaudio playback associated with the video playback operation. The usermay, if desired, configure whether device 10 pauses the audio.

In mode 84, device 10 is in a pause standby mode. In pause standby mode84, device 10 may dim display 16 to a pause standby brightness level(e.g., a partial standby brightness level) to conserve power. The videoplayback operation of mode 80 may be paused while device 10 is in mode84. If desired, device 10 may place components such as processingcircuitry and wireless transceiver circuitry in a standby mode whiledevice 10 is in mode 84 (e.g., by turning off unused CPU cores orreducing clock rates).

With one suitable arrangement, device 10 may be performing gazedetection operations while in pause standby mode 84. For example, device10 may capture images using camera 30 at regular intervals and mayanalyze the images using gaze detection software to continually monitorwhether the user's gaze has returned to device 10 and display 16.

Device 10 may switch from pause standby mode 84 to mode 80 wheneverappropriate. For example, whenever device 10 detects that a user's gazeis once again directed towards display 16, device 10 may enter an activemode such as mode 80 (e.g., as illustrated by line 86), brighten display16 to the active brightness level, and resume the video playbackoperation. Device 10 may also enter mode 80 when device 10 detectsactivity such as user activity received through a button such as button19 or network activity received through a wired or wirelesscommunications link (e.g., as illustrated by dashed line 87). Ingeneral, device 10 will enter mode 80 whenever a user resumesinteracting with device 10.

Because device 10 enters mode 80 when it detects that the user's gaze isdirected towards display 16 (e.g., as illustrated by line 86), the userof device 10 need not press a button or provide other input to awakendevice 10 from the pause standby state and resume the video playbackoperation of mode 80. Instead, device 10 can automatically awaken itself(e.g., switch to mode 80) and resume the video playback operation whenthe user directs their gaze towards display 16.

If desired, device 10 may operate in a standby mode such as standby mode90 in which display 16 is turned off. For example, when device 10 isoperating in pause standby mode 84 and no user activity is detected fora given period of time (e.g., within a period of time such as onesecond, two seconds, . . . , ten seconds, twenty seconds, thirtyseconds, etc.), device 10 may enter standby mode 90 and turn off display16. Because standby mode 90 involves a lower power state for device 10then pause standby mode 84, mode 90 may sometimes referred to as fullstandby mode. As illustrated by line 88 in FIG. 6, device 10 may enterfull standby mode 90 after device 10 detects that the user has lookedaway (e.g., as illustrated by line 82) and after a given period of userinactivity has elapsed following the device's detection that the userlooked away.

In standby mode 90, device 10 may operate with display 16 turned off.Device 10 may also place other suitable components into standby (e.g.,wireless circuitry, etc.).

With the arrangement of FIG. 6, as long as device 10 detects that theuser's gaze is directed at the device (e.g., the user is looking atdisplay 16), device 10 may remain in active mode 80 and video playbackoperation can continue (e.g., until the video is completed or theoperation is stopped). Device 10 may remain in mode 80 even when noother user activity is being received (e.g., when the user is notpressing a button such as button 19 or providing user input through atouch screen such as touch screen display 16). This type of arrangementmay be beneficial when a user is viewing a video on display 16 of device10 without providing user input and would be inconvenienced if device 10were to attempt to conserve power by dimming the video screen. Device 10can pause the video playback operation when the user temporarily looksaway and can then resume operation when the user returns their gaze todevice 10. This allows the user of device 10 to automatically pause avideo without having to provide direct user input (e.g., withoutselecting a pause button). The video can be paused simply by lookingaway from a video display such as display 16.

If desired, device 10 may continue to perform gaze detection operationswhen operating in standby mode 90. As illustrated by dashed line 93,device 10 may switch from standby mode 90 to active mode 80 and resumethe video playback operation of mode 80 when device 10 detects that auser's gaze is again directed towards display 16.

As illustrated by line 92, when device 10 detects activity, device 10may switch from mode 90 to active mode 80 (e.g., device 10 may turn ondisplay 16 to the active brightness level). As an example, device 10 mayenter active mode 80 in response to user activity such as button pressactivity received through a button such as button 19 and in response toother activity such as network activity (e.g., activity received througha wired or wireless communications link).

If desired, device 10 may automatically resume a video playbackoperation when the device switches to active mode 80 from a standby modesuch as pause standby mode 84 or full standby mode 90. With anothersuitable arrangement, device 10 may present the user with an option suchas an on-screen selectable option to resume the video playback operationwhen the device switches to active mode 80.

Device 10 may have touch screen capabilities and may implement a powermanagement scheme using gaze detection capabilities to control thedevice's touch screen capabilities. With this type of scheme, which isillustrated by FIG. 7, device 10 can switch between an active mode, apartial standby mode, and a standby mode.

Touch screen functions can be adjusted to conserve power. For example,display 16 may be a touch screen display that can operate at varyingspeeds (e.g., a fast speed and a slow speed) or with varying levels offunctionality (e.g., general touch sensitivity, localized touchsensitivity, and gesture-capable touch sensitivity). These features canbe adjusted based on gaze detection data.

With one suitable arrangement, touch screen display 16 may operate at afirst frequency (e.g., at a relatively high speed) when device 10 is inactive mode 94 and a second frequency (e.g., a relatively low speed)when device 10 is in standby mode 104. The frequency of touch screendisplay 16 may be the frequency at which the touch screen scans for userinput (e.g., once every 10 milliseconds, 50 milliseconds, 100milliseconds, 200 milliseconds, etc.).

If desired, touch screen display 16 may operate at a first level offunctionality when device 10 is in mode 94 and at a second level offunctionality when device 10 is in mode 104. For example, when device 10is in active mode 94, touch screen display 16 may be configured to sensethe location of user input within the area of display 16. Device 10 mayalso be configured to sense user inputs such as multi-touch user inputsand gestures such as swipe gestures and swipe and hold gestures while inmode 94. In contrast, when device 10 is in standby mode 104, touchscreen display 16 may be configured such that display 16 can sensegeneral user input such as the presence or absence of contact withoutbeing able to resolve the location of the input. The power consumptionof display 16 may be reduced when display 16 is configured in this way.

In mode 94, device 10 is in an active mode. While device 10 is in theactive mode, device 10 may perform gaze detection operations. Touchscreen display 16 may be operating at a relatively high frequency (e.g.,in the high power mode) while device 10 is in active mode 94. Withanother suitable arrangement, touch screen display 16 may be operatingat or near its maximum capability (e.g., touch screen display 16 may beconfigured to sense the location of user inputs and to sense user inputssuch as multi-touch inputs and gestures). Display 16 may also bedisplaying an application display screen (e.g., a home page, a telephoneapplication information page, a media player screen, etc.) at an activebrightness level.

When device 10 detects that the user has looked away from display 16(e.g., using a gaze detection sensor such as camera 30), device 10 maydim display 16 and enter partial standby mode 98, as illustrated by line96.

In mode 98, device 10 is in a partial standby mode. In partial standbymode 98, device 10 may dim display 16 to a partial standby brightnesslevel to conserve power and may retain the touch screen capabilities ofdisplay 16. (Alternatively, touch screen capabilities can be reduced inmode 98.)

Device 10 may switch from partial standby mode 98 to active mode 94whenever appropriate. For example, when device 10 detects that a user'sgaze is directed towards display 16, device 10 may enter an active modesuch as mode 94 (e.g., as illustrated by line 100) and may brightendisplay 16 to the active brightness level. Device 10 may also enteractive mode 94 when device 10 detects user activity (e.g., asillustrated by dashed line 99). In arrangements in which the touchscreen capabilities of display 16 remain at the active mode level whendevice 10 is in mode 98, display 16 may be able to receive locationspecific user inputs (e.g., inputs specific to a particular portion ofdisplay 16) while device 10 is in mode 98.

If desired, device 10 may operate in a full standby mode such as standbymode 104 in which display 16 is turned off and the touch screencapabilities of display 16 are reduced. As an example, when device 10 isoperating in partial standby mode 98 and no user activity is detectedfor a given period of time, device 10 may enter standby mode 104. Device10 may enter standby mode 104 as illustrated by line 102 after device 10detects that the user has looked away (e.g., as illustrated by line 96)and after a given period of user inactivity has elapsed following thedevice's detection that the user has looked away.

With another suitable arrangement, device 10 may enter standby mode 104directly from active mode 94 when no user activity is detected for aconfigurable period of time (e.g., as illustrated by dashed line 108).Device 10 may enter standby mode 104 even when device 10 detects that auser's gaze is directed towards display 16. If desired, the time periodof user inactivity required before device 10 enters mode 104 directlyfrom mode 94 (e.g., when a user's gaze is still directed towards device10) may be longer than the time period of user inactivity requiredbefore device 10 enters mode 104 from mode 98 (e.g., when the user'sgaze is not directed towards device 10). For example, the inactivityperiod associated with the mode transition of line 108 may be one minuteor more while the inactivity period associated with the mode transitionof line 102 may be thirty seconds or less.

In standby mode 104, device 10 may operate with a display portion ofdisplay 16 turned off. The display portion of display 16 and a touchscreen portion of display 16 may be powered and configuredindependently. In mode 104, device 10 may reduce the touch screencapabilities of the touch screen portion of display 16 (e.g., byreducing the frequency at which touch screen display 16 scans for userinput, by configuring display 16 such that user inputs can only besensed generally, by disabling the touch screen capabilities of display16, etc.).

If desired, device 10 may continue to perform gaze detection operationswhen operating in standby mode 104. As illustrated by dashed line 105,device 10 may switch from standby mode 104 to active mode 94 when device10 detects that a user's gaze is directed towards display 16.

As illustrated by line 106, device 10 may also switch from mode 104 toactive mode 94 when activity is detected (e.g., device 10 may turn ondisplay 16 to the active brightness level and restore the touch screencapabilities of display 16 to the active capability level).

If desired, power can be further conserved by reducing the powerconsumption of components such as a processor, wireless communicationscircuitry, etc. while in full standby mode 104 and/or partial standbymode 98. For example, when device 10 is placed in full standby mode 104or partial standby mode 98, the clock frequency for the clock that isused to operate processing circuitry 36 (e.g., a microprocessor) may bereduced. The number of processor cores that are active in processingcircuitry 36 may also be reduced. Some or all of wireless communicationscircuitry 44 may be placed in a low-power state or turned off. Theamount of additional circuitry that is powered down when device 10enters modes 98 and 104 may be the same or, if desired, relatively morecircuitry may be powered down in full standby mode 104 than in partialstandby mode 98.

In configurations in which device 10 has additional components, some orall of these components can be selectively powered down. Device 10 mayhave additional power down modes in which different numbers of thesecomponents have been placed in low-power states. Any suitable criteriamay be used to determine when to switch device 10 between these modes.For example, gaze detection data, user input data, and/or sensor datamay be used to determine an appropriate mode in which to operate device10. Components that may be powered down in this way include proximitysensors, light sensors such as an ambient light sensor, cameras, motionssensors such as accelerometers, audio circuits, radio-frequencytransceiver circuitry, radio-frequency amplifiers, audio amplifiers,serial and parallel port communications circuits, thermal sensors,touch-screen input devices, etc.

As illustrated by FIG. 8, device 10 can implement a power managementscheme in which gaze detection circuitry is turned on or off or isotherwise adjusted in real time. In the scheme illustrated by FIG. 8,device 10 can switch between an active mode, a partial standby mode, anda standby mode.

The gaze detection capabilities of device 10 can be adjusted to conservepower depending on the mode in which device 10 is operating. Forexample, device 10 may perform gaze detection operations by takingimages using camera 30 or other imaging circuitry at a first rate whilein an active mode and at a second rate that is less than the first ratewhile in a standby mode. If desired, device 10 may suspend gazedetection operations while in standby. When the gaze detectionoperations of device 10 are slowed down (e.g., performed at the secondrate) or suspended, device 10 may consume a reduced amount of power.

In mode 110, device 10 is in an active mode. While device 10 is in theactive mode, device 10 may perform gaze detection operations. Forexample, device 10 may perform gaze detection operations by takingimages at a given rate to search for a user's gaze (e.g., once every 100milliseconds, 200 milliseconds, 250 milliseconds, 500 milliseconds, 1second, 2 seconds, etc.). These images may then be analyzed to determinewhether the user of device 10 is looking at device 10. Display 16 maysimultaneously display an application display screen (e.g., a home page,a telephone application information page, a media player screen, etc.)at an active brightness level.

When device 10 detects that the user has looked away from display 16(e.g., using a gaze detection sensor such as camera 30), device 10 maydim display 16 and enter partial standby mode 114 as illustrated by line112.

In mode 114, device 10 is in a partial standby mode. In partial standbymode 114, device 10 may dim display 16 to a partial standby brightnesslevel to conserve power. If desired, device 10 may also reduce the speedat which images are captured for gaze detection operations in device 10(e.g., to a lower multiple of the rate at which gaze detection imageswere captured in mode 110 such as one-half, one-quarter, etc. of therate in mode 110).

Device 10 may switch from partial standby mode 114 to active mode 110whenever appropriate. For example, when device 10 detects that a user'sgaze is directed towards display 16, device 10 may enter an active modesuch as mode 110 (e.g., as illustrated by line 116) and may brightendisplay 16 to the active brightness level. Device 10 may also enteractive mode 110 when device 10 detects user activity (e.g., asillustrated by line 118).

If desired, device 10 may operate in a full standby mode such as standbymode 122 in which display 16 is turned off and the gaze detectioncapabilities of device 10 are also turned off (e.g., camera 30 is turnedoff). As an example, when device 10 is operating in partial standby mode114 and no user activity is detected for a given period of time, device10 may enter standby mode 122. Device 10 may enter standby mode 122 asillustrated by line 120 after device 10 detects that the user has lookedaway (e.g., as illustrated by line 116) and after a given period of userinactivity has elapsed following the device's detection that the userhas looked away.

In standby mode 122, device 10 may operate with display 16 turned offand with gaze detection disabled (e.g., turned off). Other circuitry mayalso be placed in a low-power standby mode (e.g., processing circuitry).

As illustrated by dashed line 124, when device 10 detects activity,device 10 may switch from mode 122 to active mode 110 (e.g., device 10may turn on display 16 to the active brightness level and turn on gazedetection capabilities to determine is a user's gaze is directed towardsdisplay 16).

With one suitable arrangement, when device 10 detects activity such asuser activity, the period of user inactivity detected by device 10 andassociated with the mode transition of line 120 may be reset. Forexample, when device 10 switches from mode 122 to active mode 110 anddetermines that the user's gaze is not directed towards display 16,device 10 may switch to mode 114 and the given period of user inactivityassociated with the mode transition of line 120 may begin anew.

The motion of device 10 can be indicative of whether device 10 is beingused by a user. If desired, device 10 may use data from an accelerometeror other motion sensor in selecting its mode of operation. For example,when device 10 detects motion above a threshold level with anaccelerometer, device 10 may activate gaze detection operations todetermine if a user is looking at the device. Device 10 may turn on gazedetection circuitry or may temporarily activate gaze detectionoperations for a given period of time (e.g., one second, five seconds,etc.) whenever a motion sensor such as an accelerometer detects that auser is shaking device 10 or device 10 is otherwise in motion. With thistype of arrangement, device 10 may be in standby mode. When device 10 ispicked up by a user, device 10 may detect that the device is in motionusing the accelerometer. Device 10 may then activate gaze detectionoperations and, if the user's gaze is properly detected, may switch toan active mode such as mode 68 in which display 16 is turned on.

Device 10 may also suspend gaze detection operations when appropriate.For example, when device 10 is receiving user input through aninput-output device 38 (e.g., when a user is providing user inputthrough one or more user input devices) or when device 10 has recentlyreceived user input, gaze detection operations may be suspended (e.g.,camera 30 may be turned off and the execution of gaze detection softwaremay be stopped). In this situation, the presence of user interfaceactivity makes it unnecessary to expend extra power operating the gazedetection circuitry.

As illustrated by FIG. 9, device 10 may also use information fromenvironmental sensors such as proximity sensors and ambient lightsensors to determine whether or not to perform gaze detectionoperations. Environmental sensors such as these may, if desired, be usedin conjunction with an environmental sensor such as an accelerometerthat detects device motion.

When device 10 is performing gaze detection operations (e.g., whendevice 10 is operating in a mode such as mode 126), device 10 maysuspend gaze detection operations whenever a sensor in device 10indicates that gaze detection operations are inappropriate or not needed(e.g., as illustrated by line 128). With one suitable arrangement,device 10 may be able to detect when gaze detection sensors such ascamera 30 would be incapable of detecting a user's gaze due to excessivevibration detected by an accelerometer. For example, device 10 maysuspend gaze detection operations (e.g., device 10 may switch tooperating in mode 130) in response to signals from the accelerometer indevice 10 that indicate the device is shaking or otherwise movingrapidly. In this example, device 10 may switch to mode 130 when theaccelerometer detects that the acceleration of device 10 exceeds a giventhreshold level. In another example, device 10 may be able to detect,using a proximity sensor, that gaze detection operations areinappropriate because an object is in close proximity to device 10 andis blocking the device's gaze detection sensors (e.g., such as when auser places device 10 against their ear and thereby blocks camera 30).If desired, device 10 may suspend gaze detection operations when anambient light sensor detects that there is insufficient light in theenvironment around device 10 for a camera such as camera 30 to captureimages in which a user's gaze could be detected. Device 10 may alsodeactivate a camera associated with gaze detection operations andsuspend a gaze detection application running on circuitry 36 when datafrom one or more sensors in device 10 indicate that gaze detectionoperations are inappropriate or wasteful of power.

When device 10 detects that gaze detection operations may be appropriate(e.g., after the sensors no longer indicate that gaze detectionoperations are inappropriate), device 10 may resume gaze detectionoperations in mode 126, as illustrated by line 132. This type ofarrangement may help device 10 to avoid performing gaze detectionoperations at inappropriate times, while ensuring that the powerconserving functionality of the gaze detection circuitry is retainedduring normal device operation.

The gaze detection capabilities of device 10 may, if desired, includevisual user identification capabilities (e.g., face recognition). Inthis type of arrangement, device 10 may distinguish between authorizedusers and unauthorized users based on image sensor data. For example,device 10 may recognize an authorized user and may unlock itselfwhenever the authorized user is detected by the device's gaze detectioncircuitry (e.g., camera 30). If desired, when device 10 detects that theauthorized user's gaze has been diverted from device 10, device 10 maylock itself to prevent unauthorized users from using device 10. Thistype of user-specific gaze detection functionality may be used for allgaze detection operations if desired. By making gaze detection specificto particular users, device 10 will not inadvertently transition fromstandby mode to active mode if a person in the user's vicinity happensto glance at the user's device.

FIG. 10 shows steps involved in processing a command to reduce the powerconsumption of display 16. Power reduction commands may be processed bydevice 10 based on gaze detection data or any other suitable data.

As show by step 134, processing may begin with reception of a powerreduction command by the processing circuitry of device 10.

Display 16 may be an OLED display or other display that has pixels thatmay be controlled individually. As shown by box 136, in this type ofsituation, device 10 may make partial or full power reduction to some orall of the pixels of display 16 in response to the received powerreduction command.

Display 16 may also be formed from a panel subsystem and a backlightsubsystem. For example, display 16 may have a liquid crystal display(LCD) panel subsystem and a light emitting diode or fluorescent tubebacklight subsystem. In backlight subsystems that contain individuallycontrollable elements such as light emitting diodes, the brightness ofthe backlight elements may be selectively controlled. For example, asshown in step 138, the brightness of some of the backlight elements maybe reduced while the other backlight elements remain fully powered.

In backlight subsystems that contain a single backlight element, thepower of the single element may be partially or fully reduced to reducepower consumption (step 140).

During the operations of steps 138 and 140, further power reductions maybe made by adjusting circuitry that controls the LCD panel subsystem.

The foregoing is merely illustrative of the principles of this inventionand various modifications can be made by those skilled in the artwithout departing from the scope and spirit of the invention.

1. A method for minimizing power consumption in a portable electronicdevice that has a display and gaze detection circuitry, the methodcomprising: performing a video playback operation in which video isdisplayed on the display; with the gaze detection circuitry, determiningwhether a user's gaze is directed towards the portable electronicdevice; and when it is determined that the user's gaze is not directedtowards the portable electronic device, pausing the video playbackoperation.
 2. The method defined in claim 1 wherein determining whetherthe user's gaze is directed towards the portable electronic devicecomprises determining whether the user's gaze is directed towards thedisplay.
 3. The method defined in claim 1 further comprising: after ithas been determined that the user's gaze is not directed towards theportable electronic device, determining whether the user's gaze hasreturned to the portable electronic device; and when it has beendetermined that the user's gaze has returned to the portable electronicdevice, resuming the video playback operation.
 4. The method defined inclaim 1 further comprising: after it has been determined that the user'sgaze is not directed towards the portable electronic device, determiningwhether the user's gaze has returned to the portable electronic device;and when it has been determined that the user's gaze has returned to theportable electronic device, presenting an on-screen selectable option toresume the video playback operation.
 5. The method defined in claim 1further comprising: when it has been determined that the user's gaze isnot directed towards the portable electronic device, placing the displayin an operating mode that reduces power consumption by the display. 6.The method defined in claim 5 wherein the display is at a firstbrightness level during the video playback operation and wherein placingthe display in the operating mode comprises reducing the brightness ofthe display to a second brightness level.
 7. The method defined in claim6 further comprising: after it has been determined that the user's gazeis not directed towards the portable electronic device, determiningwhether the user's gaze has returned to the portable electronic device;and when it has been determined that the user's gaze has returned to theportable electronic device, increasing the brightness of the display tothe first brightness level.
 8. The method defined in claim 5 whereinplacing the display in the operating mode that reduces power consumptioncomprises turning off the display.
 9. A method for using a portableelectronic device having an accelerometer and gaze detection circuitry,the method comprising: with the accelerometer, determining whether ameasured acceleration level for the portable electronic device hasexceeded a given threshold value; and when it has been determined thatthe acceleration level of the portable electronic device has exceededthe given threshold value, disabling the gaze detection circuitry. 10.The method defined in claim 9 further comprising: after it has beendetermined that the acceleration level of the portable electronic devicehas exceeded the given threshold value, determining whether theacceleration level of the portable electronic device has dropped below asecond threshold value; and when it has been determined that theacceleration level of the portable electronic device has dropped belowthe second threshold value, enabling the gaze detection circuitry. 11.The method defined in claim 9 wherein the gaze detection circuitrycomprises a camera that is used to determine whether the user's gaze isdirected towards the portable electronic device and wherein disablingthe gaze detection circuitry comprises turning off the camera.
 12. Aportable electronic device comprising: an image sensor that capturesimages of a user; and circuitry that processes the captured images todetermine whether the user is looking at the portable electronic device,wherein the circuitry is configured to suspend captured image processingoperations when user activity with the portable electronic device isdetected.
 13. The portable electronic device defined in claim 12 whereinthe circuitry comprises at least one button and wherein the useractivity comprises presses of the button by the user.
 14. The portableelectronic device defined in claim 12 wherein the user activity includesmovement of the portable electronic device and wherein the circuitrycomprises a sensor that detects the movement of the portable electronicdevice.
 15. The portable electronic device defined in claim 12 furthercomprising: a proximity sensor that detects whether any objects arewithin a given distance of the portable electronic device, wherein thecircuitry is configured to suspend captured image processing operationswhen the proximity sensor detects an object within the given distance.16. The portable electronic device defined in claim 12 furthercomprising: a light sensor that measures a brightness level of ambientlight, wherein the circuitry is configured to suspend captured imageprocessing operations when the measured brightness level of ambientlight is less than a given brightness level.
 17. A portable electronicdevice comprising: a display; an image sensor that captures images of auser; a user input device that receives input from the user; andcircuitry that processes the captured images to determine whether theuser is looking at the portable electronic device, wherein the circuitryis configured to: power the display at a first brightness level when itis determined that the user is looking at the portable electronicdevice; and turn off the display after a given period of time haselapsed in which no input has been received from the user by the userinput device, wherein the given period of time begins when it isdetermined that the user has looked away from the portable electronicdevice.
 18. The portable electronic device defined in claim 17 whereinthe circuitry is configured to reduce power to the display so that thedisplay is at a second brightness level during the given period of timein which no input has been received from the user by the user inputdevice.
 19. The portable electronic device defined in claim 17 whereinthe circuitry is configured to once again power the display at the firstbrightness level when the user input device receives input from theuser.
 20. The portable electronic device defined in claim 17 wherein theuser input device comprises at least one button and wherein the inputfrom the user comprises presses of the button by the user.
 21. Theportable electronic device defined in claim 17 wherein the circuitry isconfigured to suspend captured image processing operations after thegiven period of time has elapsed.
 22. The portable electronic devicedefined in claim 21 wherein the circuitry is configured to resumecaptured image processing operations when the user input device receivesinput from the user.
 23. A method for using a portable electronic devicehaving a display, a user input device, and gaze detection circuitry, themethod comprising: with the gaze detection circuitry, determiningwhether a user's gaze is directed towards the portable electronicdevice; when it is determined that the user's gaze is not directedtowards the portable electronic device, reducing power to the display;and when the user input device receives input from a user and power tothe display has been reduced, increasing power to the display so thatthe display is at a first brightness level.
 24. The method defined inclaim 23 wherein reducing power to the display comprises reducing powerto the display so that the display is at a second brightness level thatis less than the first brightness level.
 25. The method defined in claim23 wherein reducing power to the display comprises turning off thedisplay.
 26. The method defined in claim 23 wherein reducing power tothe display comprises reducing power to the display so that the displayis at a second brightness level that is less than the first brightnesslevel, the method further comprising: after a given period of time haselapsed in which no input has been received by the user input device,turning off the display.
 27. A portable electronic device comprising: adisplay; an image sensor that captures images of a user; and circuitrythat processes the captured images to determine whether the user islooking at the portable electronic device, wherein the portableelectronic device is configured to operate in: an active mode in whichthe display is powered at a first brightness level; a partial standbymode in which the display is powered at a second brightness level thatis less than the first brightness level; and a full standby mode inwhich the display screen and at least part of the circuitry thatprocesses the captured images are powered down.
 28. The portableelectronic device defined in claim 27 wherein the portable electronicdevice is further configured to switch from operating in the active modeto operating in the partial standby mode when the circuitry determinesthat the user has looked away from the portable electronic device. 29.The portable electronic device defined in claim 27 wherein the portableelectronic device is further configured to switch from operating in thepartial standby mode to operating in the full standby mode after a givenperiod of time has elapsed in which no input has been received from theuser by the user input device.
 29. A portable electronic devicecomprising: a display; an image sensor that captures images of a user; aprocessor; and circuitry that processes the captured images to determinewhether the user is looking at the portable electronic device, whereinthe portable electronic device is configured to operate in: a first modein which the display is powered at a first brightness level; and asecond mode in which power to the display and the processor is reduced.30. The portable electronic device defined in claim 29 wherein the firstmode is an active mode, wherein the second mode is a full standby modein the display is powered down, and wherein the portable electronicdevice is further configured to operate in a partial standby mode inwhich the display is powered at a second brightness level that is lessthan the first brightness level.
 31. The portable electronic devicedefined in claim 29 wherein the processor includes a given number ofprocessing cores, wherein, in the first mode, the processor isconfigured to operate using the given number of the processing cores,and wherein, in the second mode, the processor is configured to operateusing fewer than the given number of the processing cores.
 32. Theportable electronic device defined in claim 29 wherein the processor isconfigured to receive a first clock in the first mode and a second clockin the second mode and wherein the first clock has a frequency that isgreater than the second clock.